Single-image super-resolution (SI-SR [1, 2]) has been an important focus of research during the last years, arguably due to the following two reasons: (1) in contrast with classic reconstruction-based super-resolution (RB-SR [3, 4]), the use of more advanced image priors in SI-SR alleviates the requirement of having different captures of the same scene with subpixel shifts; and (2) numerical stability problems that might arise when reconstructing the super-resolved image under inaccurate models are also avoided by using more meaningful image priors.
Even though the above two advantages imply great practical benefits for many applications, noise handling is an often overlooked yet major shortcoming of SI-SR techniques. The use of different captures of the same scene when estimating a super-resolved image in RB-SR naturally introduces an implicit reduction of the noise level and an improvement of the problem condition. However, most SI-SR techniques will produce an increased noise level in the upscaled image.
Cross-scale self-similarity, introduced by Glasner et al. [2], has been a widely utilized prior for SI-SR algorithms. Its main advantage is the adaptivity to the image contents. Whereas this type of prior was originally presented while considering all possible locations within all possible scales as references, posterior results—such as those provided by Freedman et al. [5], where the search range for examples is restricted to a small rectangular window in a single scale, or Yang et al. [6], where conditions are determined for further reducing the search space to a single location (in-place example) when combined with linear regression—have progressively allowed to improve the efficiency of SR approaches without losing accuracy. An often omitted question is the extension of these powerful priors to the case of noisy images. This problem has been recently addressed by Zontak et al. [7] in a denoising scenario.
Thus, various single-image super-resolution algorithms are known that provide good results as long as the input image is noise free. A problem is that these algorithms provide much worse results as soon as the input image is noisy. The usage of in-place optimization with patches from different downscaled versions of noisy images for denoising and performing super-resolution (no HF/LF) is known from [4].
It is noted that the actual goal of upscaling/SR is to preserve, at a higher resolution, the quality that an image has at the original scale. This is relatively easy to achieve if the image is clean, but if it is not, then denoising has to be applied before upscaling in order to avoid amplifying the noise power. In classical reconstruction-based SR with several images, it is known to jointly apply denoising with the upscaling step. For example-based SR (i.e. single image super-resolution, SI-SR), a one-step upscaling and denoising has been disclosed by Yang et al. [6], where in-place self-similarity combined with offline learning is used and where noise-robustness is explicitly demonstrated. However, [6] relies on and is limited to in-place self-similarity.